Power calculations for mouse research

Thomas G. Stewart, PhD
13 August 2019

These slides were prepared for the 2019 Vanderbilt Mouse Kidney Injury Workshop (link) on 13 August 2019.

Slides



These slides are available on my faculty page under Teaching


tgstewart.xyz

The goals of this session are to

  • provide a big picture overview of power calculations
  • explain when power calculations are (and are not) appropriate
  • introduce online resources for power calculations
  • work through hands-on examples of calculating power

The big picture

The big picture

The big picture <font color="red">for an hypothesis driven, confirmatory study</font>

How does this compare to a pilot study or an exploratory study?

Pilot study

Important side note

As an example, consider a comparison of means between two groups.

plot of chunk unnamed-chunk-2

What is the interpretation if \( \ p < 0.05 \)?

What is the interpretation if \( \ p > 0.05 \)?

Important side note

Important side note

It is a mistake to say: There was no difference in mean Y between groups.

It is correct to say: At the given sample size, a difference in mean Y was not detected.

It is cutting-edge to use a second-generation p-value and to prespecify a meaningful null region.




Important side note over

Back to the big picture

 

Big picture

Suppose one knew the truth

Big picture

Suppose one repeated the study multiple times

Would one get the same result each time?




Probably not

Key vocabulary

Operating characteristic:

The distribution of a study trait over repeated executions of the study.

Types of study traits

  1. Sample size
  2. False positive rate
  3. False negative rate
  4. False direction rate

Possible study traits

  1. Sample size
  2. False positive rate (Type I error)
  3. False negative rate (Type II error)
  4. False direction rate (Type S error)

Type I error

Type I error

Type I error

Type II error

Type II error

Type II error

Type II error

Operating characteristics



Who cares about operating characteristics?

A researcher should understand the sample size needed for a study to generate conclusive results.

Operating characteristics are helpful quantities for grant reviewers and funding agencies and anyone who must judge the likelihood that a study will generate conclusive results within budget.

Data scientists calculate operating characteristics to understand the performance of a proposed analysis method.

Operating characteristics



When should we care about power?

Grant reviewers generally prefer to fund projects that will lead to conclusive results.

Data scientists prefer that study analysis plans use methods that maximize power.

Operating characteristics



When should we NOT care about power?

Grant reviewers generally prefer to fund projects that will lead to conclusive results.

Data scientists prefer that study analysis plans use methods that maximize power.